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ABSTRACT 

This  paper  proposes  a  systems  engineering  process  utiliz¬ 
ing  the  conceptual  artifacts  of  the  Model  Driven  Architec¬ 
ture  (MDA)  describing  platform  independent  views  of 
models  to  capture  operational  requirements,  to  derive  es¬ 
sential  tasks,  and  to  combine  these  tasks  into  scenarios  and 
vignettes  with  attributed  metrics.  This  model-independent 
mission  description  is  then  used  to  identify  supporting  si¬ 
mulation  services  that  implement  the  identified  military 
means  and  capabilities  to  perform  the  tasks  in  the  given 
context.  Once  the  services  are  identified,  the  necessary  si¬ 
mulation  middleware  to  federate  the  services  is  identified 
and  the  interfaces  are  configured  using  the  technical  arti¬ 
facts  of  the  MDA  describing  platform  specific  views  of 
systems.  This  systems  engineering  process  provided  sup¬ 
port  for  simulation  development  for  the  US  Army’s  Pro¬ 
gram  Executive  Office  -  Soldier. 

1  INTRODUCTION 

Old  Dominion  University  (ODU)  and  the  Virginia  Model¬ 
ing  Analysis  and  Simulation  Center  (VMASC)  support  the 
US  Army  with  expertise  on  systems  engineering  processes, 
in  particular  on  the  use  of  principles  of  the  Model  Driven 
Architecture  (MDA)  supporting  simulation  system  interop¬ 
erability.  In  a  recent  project  conducted  in  collaboration 
with  the  United  States  Military  Academy,  a  systems  engi¬ 
neering  process  was  developed  that  utilizes  the  artifacts  of 
MDA  to  support  building  federations  driven  by  operational 
requirements.  This  paper  documents  the  process,  gives  an 
example,  and  summarizes  some  necessary  requirements  to 
apply  this  process  to  align  acquisition,  development,  test¬ 
ing,  training,  and  operational  support  for  the  armed  forces. 

The  systems  engineering  process  proposed  in  this  pa¬ 
per  is  based  on  several  relevant  and  community  accepted 
methods  and  standards.  In  the  second  section  of  this  paper, 
these  methods  are  reviewed  in  order  to  root  the  proposed 
process  in  already  accepted  work.  At  the  same  time,  alter¬ 
native  views,  that  are  currently  often  perceived  to  be  com¬ 


peting  alternatives,  are  supported  by  a  common  process. 
Due  to  the  enormous  variety  of  supporting  methods  and 
standards,  the  section  can  be  neither  complete  nor  exclu¬ 
sive.  The  documented  principles  should  support  extending 
this  to  other  alternatives  as  well,  and  the  authors  welcome 
related  discussions. 

The  third  section  shows  how  the  artifacts  of  MDA  can 
be  used  to  unify  the  different  contributions  as  a  technical 
support  of  the  recommended  systems  engineering  process. 
The  components  of  Computer  Independent  Models  (CIM) 
and  Platform  Independent  Models  (PIM)  are  used  to  model 
mission  essential  task  and  compose  those  into  vignettes 
and  scenarios.  The  resulting  elements  describe  a  task  that 
needs  to  be  performed  in  a  given  operational  context.  In 
addition,  metrics  are  assigned  that  measures  how  well  a 
system  with  the  required  capability  fulfills  this  task  in  the 
given  context.  Next,  the  selection  of  simulation  services  is 
conducted  based  on  these  vignettes.  In  order  to  be  able  to 
support  this,  the  simulation  service  capabilities  themselves 
must  be  specified  as  PIM  as  well. 

Once  the  project  manager  -  or  his  supporting  technical 
advisors  -  decides  which  system  will  support  the  evalua¬ 
tion  with  which  capability,  the  work  can  be  transformed  to 
the  technical  level.  The  selected  simulation  services  need 
to  be  federated  on  available  simulation  middleware  solu¬ 
tions.  These  can  be  standardized  solutions,  such  as  the 
Runtime  Infrastructure  (RTI)  of  the  High  Level  Architec¬ 
ture  (HLA)  or  the  use  of  Protocol  Data  Units  (PDU)  as  de¬ 
fined  in  the  Distributed  Interactive  Simulation  (DIS)  stan¬ 
dards,  or  industry  standards,  such  as  Web  services.  In 
addition,  the  mapping  to  very  efficient  special  solutions, 
such  as  binary  links  in  radio-based  communications,  is 
possible  as  well,  although  this  seems  to  be  the  exception. 
Once  the  technical  protocols  are  selected,  the  Platform 
Specific  Model  (PSM)  artifacts  of  MDA  can  be  applied  to 
generate  the  necessary  interfaces,  mapping,  and  connec¬ 
tions.  Additional  support  can  be  provided  when  Base  Ob¬ 
ject  Models  (BOM)  are  used. 

The  ideas  were  used  in  support  of  an  acquisition  task 
currently  conducted  by  the  US  Army.  The  Army's  Program 
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Executive  Office  (PEO)  -  Soldier  has  the  complex  task  of 
acquiring  and  integrating  a  system  of  soldier  equipment 
that  meets  their  mission  requirements.  In  order  to  better  as¬ 
sess  trade-offs  in  different  soldier  architectures,  they  seek 
an  improved  simulation  capability  that  better  represents  the 
individual  soldier  on  the  battlefield.  No  single  model  pro¬ 
vides  this  capability.  They  are  pursuing  a  strategy  of  inte¬ 
grating  three  different  simulation  models  to  take  advantage 
of  the  strengths  of  each.  In  section  four,  we  show  how  the 
definition  of  questions  in  support  of  acquisition  decisions 
is  supported  by  the  systems  engineering  process  using  the 
MDA  artifacts.  The  examples  given  here  are  only  a  small 
subset,  but  the  principle  can  be  shown. 

In  section  five,  the  idea  is  generalized  beyond  the  ex¬ 
ample.  Currently,  acquisition,  development,  testing,  train¬ 
ing,  and  operational  support  are  only  loosely  coupled.  The 
approach  recommended  in  this  paper  allows  the  reuse  of 
significant  findings,  operational  requirements,  and  con¬ 
straints  bridging  the  phases  of  the  life  cycle  of  a  system. 
This  results  potentially  in  better  aligned  support  for  the 
warfighters  needs.  The  supported  project  shows  the  feasi¬ 
bility  of  these  recommendations. 

2  RELEVANT  METHODS  AND  STANDARDS 

The  necessity  of  applying  systems  engineering  processes  in 
support  of  system  decisions  in  all  phases  of  the  life  cycle  is 
nothing  new.  Also,  to  anchor  such  processes  in  the  opera¬ 
tional  necessities  defined  by  requirements  is  common  pro¬ 
cedure.  What  is  innovative  is  the  idea  to  use  common  arti¬ 
facts  in  support  of  all  phases  of  the  useful  life  cycle  of 
systems  in  a  consistent  way,  covering  all  aspects  of  the  op¬ 
erational  life  cycle.  This  starts  with  the  identification  of  an 
operational  gap,  a  certain  capability  that  is  required  to  im¬ 
plement  doctrine.  Once  this  capability  is  identified,  the 
procurement  and  acquisition  community  has  to  decide  if  a 
new  system  should  be  introduced  to  deliver  the  function 
implementing  the  capability,  or  if  an  existing  system  can 
be  improved  to  provide  the  functionality. 

On  the  operational  side,  these  steps  can  be  supported 
by  the  Military  Missions  and  Means  Framework  (MMF) 
and  related  task  list  activities.  This  will  be  described  in  the 
first  subsection.  This  process  is  closely  related  to  the  task 
to  produce  operational  views  in  the  DoD  Architecture 
Framework.  The  systems  view  is  represented  by  the  sys¬ 
tems  and  capabilities  that  are  used  to  provide  the  means 
within  a  mission. 

The  technical  guidance  is  provided  by  guidance  doc¬ 
uments  and  standards.  In  the  second  subsection,  the  Fed¬ 
eration  Development  and  Execution  Process  (FEDEP)  sup¬ 
porting  HLA  and  the  more  general  counterpart  often 
applied  in  Europe,  the  Synthetic  Environment  Develop¬ 
ment  and  Exploitation  Process  (SEDEP)  will  be  used  to 
show  necessary  steps  that  need  to  be  supported  by  the  sys¬ 
tems  engineering  process.  In  addition,  the  NATO  Code  of 


Best  Practice  (NCOBP)  for  Command  and  Control  (C2) 
Assessment  gives  guidance  as  well. 

Finally,  MDA  and  the  necessary  PIMs  and  PSMs  ideas 
are  described  in  the  third  subsection.  Although  not  used  in 
the  study,  the  use  of  BOMs  has  been  identified  as  addi¬ 
tional  support.  Scope  and  resolution  of  this  overview  are 
limited  to  the  level  needed  to  understand  their  application 
in  the  following  section,  in  which  the  resulting  recom¬ 
mended  systems  engineering  process  will  be  described. 

2.1  Missions  and  Required  Capabilities 

Truly  integrated  operations  depend  on  a  solid  foundation  of 
common  elements  understood  between  all  participating 
partners  and  organizations.  The  current  approach  is  to  es¬ 
tablish  a  mission  essential  task  list  (METL)  that  lists  the 
operational  tasks  forces  need  to  perform  to  doctrinally  ac¬ 
complish  a  given  mission.  These  tasks  may  also  be  mapped 
to  a  common  Universal  Joint  Task  List  (UJTL).  Several 
separate  initiated  US  DoD  programs  as  well  as  some  Ho¬ 
meland  Security  efforts  are  planning  to  base  their  metrics 
of  performance  on  mission  essential  tasks  (MET).  Within 
NATO,  comparable  efforts  are  undertaken,  although  the 
resulting  task  lists  are  not  always  well  aligned  between  all 
nations.  Despite  the  need  for  better  harmonization,  in  all 
these  efforts  a  military  task  is  identified  and  necessary  ca¬ 
pabilities  to  perform  these  tasks  are  captured.  The  targeted 
result  is  a  list  of  mission  essential  tasks,  related  capabili¬ 
ties,  and  metrics  to  measure  the  performance.  It  should  be 
pointed  out  that  an  MET  should  not  be  tightly  coupled  with 
a  system  or  a  capability  implementation.  The  MET  should 
describe  the  conceptual  capability  which  -  at  least  in  the¬ 
ory  -  can  be  delivered  by  several  systems  or  system  com¬ 
ponents. 

These  ideas  are  tightly  connected  with  the  MMF:  the 
context  is  defined  by  a  military  mission  -  which  is  a  set  of 
MET  -  and  military  means  that  are  needed  to  conduct  the 
mission.  The  MMF  is  therefore  the  operational  view  de¬ 
scribing  what  operational  nodes  are  needed  and  which  op¬ 
erational  activities  are  conducted.  The  systems,  which  are 
normally  systems  that  have  to  be  evaluated  or  that  are  un¬ 
der  test,  provide  capabilities  that  implement  the  means 
needed  to  conduct  a  mission.  This  is  consistent  with  the 
systems  view:  how  are  missions  and  means  concretely  in¬ 
stantiated?  In  order  to  assure  scientific  evaluations  based 
on  experimentations,  a  metric  is  needed  that  captures 

(a)  what  data  is  collected,  and 

(b)  how  this  data  is  used  to  define  success  or  failure. 

In  order  to  be  able  to  conduct  the  evaluation,  these 

task  elements  must  be  put  into  a  meaningful  operational 
context.  This  is  done  by  setting  them  into  the  context  of  a 
scenario  or  a  vignette. 

The  focus  of  all  these  activities  should  be  the  evalua¬ 
tion  of  the  system  to  be  evaluated  or  under  test.  It  is  essen¬ 
tial  to  track  other  capabilities  and  their  relative  changes 
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based  on  the  system  to  be  evaluated  as  well,  in  particular 
when  it  comes  to  indirect  or  higher  order  effects,  but  the 
system  is  the  main  part  of  this  effort.  Therefore,  the  design 
process  for  setting  up  a  scenario  is  as  follows: 

1.  All  tasks  that  are  conducted  by  the  system  are 
added  to  the  task  list  to  be  evaluated 

2.  All  tasks  that  are  supported  by  tasks  conducted  by 
the  system  are  added  to  the  task  list 

3.  All  tasks  that  are  influenced  (higher  order  effects) 
by  the  system  are  added  to  the  task  list 

4.  Operational  vignettes  or  scenarios  comprising  all 
tasks  on  the  task  list  (if  necessary  prioritized  by 
operational  effects)  are  defined 

The  result  of  these  steps  is  a  scenario  or  a  list  of  vi¬ 
gnettes  that  comprises  all  tasks  and  metrics  needed  to 
evaluate  the  system. 

2.2  FEDEP,  SEDEP,  and  the  NATO  Code  of  Best 
Practice  for  C2  Assessment 

In  order  to  support  the  evaluation,  it  is  more  than  likely 
that  more  than  one  simulation  system  will  be  used.  The  se¬ 
lection  of  contributing  systems  should  be  based  on  the  si¬ 
mulated  systems,  their  capabilities,  and  their  ability  to  sup¬ 
port  the  desired  metrics.  The  NATO  Code  of  Best  Practice 
for  C2  Assessment  was  produced  in  order  to  facilitate  high 
quality  assessments.  It  identifies  several  steps  of  an  itera¬ 
tive  process: 

•  In  the  initial  phase,  the  team  starts  with  the  prob¬ 
lem  formulation  and  related  high-level  solution 
strategies.  This  corresponds  with  the  question  of 
what  the  system  to  be  evaluated  should  do  in  sup¬ 
port  of  which  missions. 

•  In  the  second  phase,  three  steps  have  to  be  con¬ 
ducted  to  refine  the  ideas  of  the  initial  phase.  In 
this  phase,  the  team  identifies  the  Human  and  Or¬ 
ganizational  Factors  (the  concepts  to  be  evalu¬ 
ated,  where  they  are,  how  they  operate,  etc.)  and 
put  them  into  the  context  of  a  Scenario.  In  addi¬ 
tion,  the  Measures  of  Merit  are  decided.  This 
phase  deals  with  identifying  the  important  con¬ 
cepts  and  processes,  their  role  in  a  scenario,  and 
how  to  measure  success  or  failure. 

•  Only  after  the  conceptualization  is  done,  the  im¬ 
plementation  phase  is  conducted.  The  selection  of 
Methods  and  Tools  -  such  as  simulation  systems 
to  use,  but  also  supporting  tools  for  the  evaluation 
-  is  one  of  the  steps.  As  important  as  the  tool  se¬ 
lection  is  to  ensure  that  the  necessary  Data  is 
available  or  can  be  obtained  within  the  constraints 
of  the  project. 

•  Finally,  Risk  and  Uncertainty  Management ,  in¬ 
cluding  sensitivity  analysis  of  proposed  solution, 
is  conducted  before  the  project  is  summarized  in 
the  deliverables. 


The  NCOBP  is  an  operations  research  process.  It  rec¬ 
ommends  best  practices  on  the  structure  of  a  project.  In  or¬ 
der  to  give  technical  guidance,  other  sources  are  needed. 
For  federation  development,  the  IEEE  1516.3  guideline  on 
the  Federation  Development  and  Execution  Process 
(FEDEP)  or  the  Euclid  RTP  11.33  description  of  the  Syn¬ 
thetic  Environment  Development  and  Exploitation  Process 
(SEDEP)  is  needed.  Both  processes  are  similar,  as  the 
FEDEP  was  used  as  a  guideline  when  the  SEDEP  was  de¬ 
veloped.  Although  the  focus  is  more  technical  than  in  the 
NCOBP,  the  necessity  to  build  a  strong  conceptual  model 
before  going  into  the  technical  details  is  emphasized  in 
both  approaches. 

•  The  SEDEP  starts  with  an  explicit  User  Needs 
Analyses  that  is  not  supported  by  the  FEDEP.  The 
following  steps  are  well  aligned,  as  the  SEDEP 
understands  itself  as  an  enhanced  FEDEP.  One  of 
the  enhancements  is  the  support  of  a  common  re¬ 
pository  for  all  produced  artifacts. 

•  The  development  process  starts  with  refinements 
of  the  user  requirements  that  lead  to  operationally 
driven  federation  system  requirements  for  the 
SEDEP.  On  the  FEDEP  side,  defining  the  federa¬ 
tion  objectives  and  developing  a  conceptual  mod¬ 
el  for  the  federation  are  the  counterparts. 

•  Based  on  this  operational  understanding,  the  fed¬ 
eration  is  designed,  implemented,  integrated,  and 
tested  in  both  process  models.  In  both  models,  the 
selection  of  federates  is  based  on  the  operational 
requirements. 

•  Finally,  the  federation  is  operated,  which  means 
that  the  federation  is  executed  and  respective  re¬ 
sults  are  prepared.  The  SEDEP  explicitly  ends  the 
process  with  performing  an  evaluation,  which  is 
kind  of  integrated  into  the  execution  phase  of  the 
FEDEP. 

Both  process  models  clearly  show  the  primary  impor¬ 
tance  of  operational  requirements.  Both  make  technical 
recommendations,  but  the  implementation  details  are  left  to 
the  model  developers. 

2.3  The  Model-Driven  Architecture 

The  Model  Driven  Architecture  (MDA)  was  developed 
under  the  lead  of  the  Object  Model  Group  (OMG)  to  facili¬ 
tate  reacting  to  business  and  technology  changes.  The  un¬ 
derlying  idea  is  to  separate  business  and  application  logic 
from  underlying  technology.  To  enable  this,  MDA  defines 
artifacts  based  on  the  Unified  Modeling  Language  (UML) 
to  describe  a  hierarchy  of  models  that  cope  with  the  vari¬ 
ous  challenges  on  different  levels. 

•  The  highest  level  of  abstraction  is  the  Computa¬ 
tion  Independent  Models  (CIM).  This  is  a  concep¬ 
tual  model  that  identifies  the  concepts  and  proc¬ 
esses  important  on  the  business  level.  This  is 
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easily  mapable  to  the  missions  and  means  identi¬ 
fied  on  the  operational  level.  The  main  artifacts 
are  use  cases. 

•  The  Platform  Independent  Models  (PIM)  capture 
concepts  and  processes  in  software  engineering 
artifacts  of  class  and  object  hierarchies,  activities, 
sequences,  and  other  means  showing  the  roles  of 
each  component.  PIM  are  very  close  to  conceptual 
models  that  already  use  vignette  and  scenario 
elements  motivating  the  various  possible  actions 
and  their  sequencing. 

•  If  this  conceptual  model  is  mapped  to  a  concrete 
platform  and  implementing  language,  middleware 
to  be  used,  etc.,  the  result  is  a  Platform  Specific 
Model  (PSM).  In  the  optimal  case,  the  PSM  can 
be  used  to  produce  code,  as  all  information 
needed  is  available. 

It  should  be  pointed  out  that  the  models  in  the  differ¬ 
ent  layers  are  not  developed  independently  from  each  oth¬ 
er.  Every  use  case  of  the  CIM  must  be  represented  in  form 
of  sequenced  actions  engaging  the  roles  as  concepts  in  the 
PIM.  The  conceptual  ideas  of  the  PIM  must  be  mapped  to 
implementing  entities,  their  capabilities  and  associations, 
and  supporting  interfaces  on  the  PSM  level.  In  theory,  this 
is  supported  by  the  use  of  defining  patterns.  If  the  support¬ 
ing  middleware  has  an  equivalent  alternative,  this  approach 
allows  switching  between  representing  PSM  without  hav¬ 
ing  to  change  the  PIM.  In  other  words:  A  federation  can  be 
implemented  using  both  middleware  approaches  alterna¬ 
tively. 

In  the  M&S  business  world,  some  M&S  middleware 
and  integration  providers  are  utilizing  this  idea  to  support 
the  migration  between  equivalent  -  or  at  least  sufficiently 
close  -  implementations,  such  as  supporting  the  Runtime 
Infrastructure  interfaces  defined  in  IEEE1516  as  well  as 
the  alternative  defined  in  version  1.3  NG  (DoD). 

2.4  Base  Object  Models 

Base  Object  Models  (BOMs)  are  a  recently  accepted  M&S 
standard  (SISO  2006).  BOMs  provide  a  key  mechanism  in 
facilitating  interoperability,  reuse,  and  composability.  The 
BOM  concept  is  based  on  the  assumption  that  piece-parts 
of  simulations  and  federations  can  be  extracted  and  reused 
as  modeling  building-blocks  or  components.  The  interplay 
within  a  simulation  or  federation  can  be  captured  and  char¬ 
acterized  in  the  form  of  reusable  patterns.  These  patterns  of 
simulation  interplay  are  sequences  of  events  between  simu¬ 
lation  elements.  What  is  of  particular  interest  in  this  paper 
is  that  the  artifacts  used  to  describe  the  behavior,  activities, 
and  interplay  between  components  -  including  the  result¬ 
ing  state  changes  in  the  sending  and  receiving  system  for 
each  interaction  -  are  UML  artifacts  describing  the  “state 
machine”  and  the  “pattern  of  interplay.” 


Although  BOMs  was  developed  in  support  of  the  High 
Level  Architecture,  their  application  is  not  limited  to  HLA 
federations.  The  metadata  produced  to  describe  the  entities, 
behaviors,  state  changes,  and  interplays  are  generally  us¬ 
able  conceptual  constructs.  In  the  scope  of  the  presented 
study,  this  means  that  simulation  components  and  simula¬ 
tion  systems  that  are  described  following  the  BOM  stan¬ 
dard  already  provide  a  significant  part  of  the  information 
required  to  support  machine-supported  selection  based  on 
operational  requirements,  as  the  BOM  description  can  be 
interpreted  as  a  “Core  PIM”  for  the  component.  Gustavson 
and  Chase  (2007)  summarize  related  ideas. 

In  this  section,  we  identified  that  the  MMF  and  METL 
support  the  operational  analysis  of  what  the  relevant  tasks 
are  when  a  system  needs  to  be  evaluated.  The  result  is  a 
description  of  tasks  in  the  context  of  vignettes  or  scenarios 
with  applicable  metrics.  Operational  requirements  should 
also  drive  technical  selection  and  integration.  The  NCOBP 
as  well  as  the  modeling  and  simulation  standards  FEDEP 
and  SEDEP  show  which  steps  are  needed  to  set  up  and  ex¬ 
ecute  a  federation.  Separating  business  logic  and  platform 
specification  leading  to  a  hierarchy  of  models  allows  the 
MDA  to  facilitate  the  migration  between  equivalent  or 
closely  related  technical  solutions.  BOM  is  a  potential 
standard  supporting  the  identification  and  selection  of  ap¬ 
plicable  components. 

In  the  next  section,  we  will  document  a  systems  engi¬ 
neering  process  that  integrates  these  ideas  enabling  the 
seamless  management  of  federation  development  for  sys¬ 
tem  evaluation  from  the  operational  analysis  to  the  techni¬ 
cal  details  of  middleware  selection  and  interface  design. 

3  THE  SYSTEMS  ENGINEERING  PROCESS 

The  systems  engineering  process  proposed  in  this  paper 
was  developed  collaboratively  by  Old  Dominion  Univer¬ 
sity  and  the  US  Military  Academy.  It  was  motivated  by  the 
need  to  support  the  project  management  with  a  consistent 
view  of  PEO  Soldier  challenges  in  compliance  with  rele¬ 
vant  processes  as  described  in  the  last  section: 

•  The  essential  tasks  to  be  used  for  the  evaluation 
should  be  identified  to  support  the  selection  or 
development  of  relevant  vignettes  or  scenarios. 

•  Simulation  systems  should  be  selected  based  on 
their  ability  to  support  the  evaluation  of  these 
tasks.  The  simulated  system  capability  should  be 
the  driver  for  the  decision. 

•  The  process  should  be  applicable  to  evaluate  al¬ 
ternatives  for  supporting  simulation  components 
and  enable  the  project  manager  to  make  informed 
decisions. 

•  The  federation  of  these  simulation  systems  should 
be  supported  utilizing  the  best  middleware  avail¬ 
able  for  the  task.  This  decision  should  be  driven 
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by  the  functionality  of  the  middleware  and  its  ne¬ 
cessity  in  the  federation  development  process. 

•  The  integration  of  systems  and  middleware  should 
be  supported  to  the  maximal  extent.  The  decisions 
of  model  integrators  should  be  reduced  to  a  mini¬ 
mum.  This  avoids  ambiguity  of  interpretations. 
Existing  solutions  should  be  reused  as  much  as 
possible. 

3.1  Identifying  Essential  Tasks 

In  evaluations,  operations  and  training,  time  and  resources 
are  always  limited.  It  is  necessary  to  concentrate  the  efforts 
on  the  essential  tasks.  For  military  operations,  the  METL 
introduced  in  section  2.1  is  a  way  to  support  the  decision 
makers  in  making  the  appropriate  selection. 

If,  for  example,  the  effect  of  a  new  type  of  body  armor 
has  to  be  evaluated,  all  tasks  in  the  METL  dealing  with 
survivability  need  to  be  evaluated.  Also  those  tasks  for 
which  the  body  armor  may  become  a  challenge,  such  as 
dismounted  mobility,  should  be  evaluated. 

The  result  is  a  list  of  tasks  in  which  the  systems  under 
evaluation  plays  significant  roles.  This  list  is  represented  in 
form  of  use  cases  that  identify  the  action  performer,  the  ac¬ 
tion  target,  and  the  action  itself.  This  use  case  list  can  be 
supported  by  storyboards  and  organizational  diagrams  of 
the  actors.  These  elements  of  the  CIM  are  represented  us¬ 
ing  UML.  This  CIM  is  the  result  of  the  first  phase. 

3.2  Setting  the  Tasks  into  Context:  Building 
Scenarios,  Vignettes,  and  Metrics 


the  common  warehouse  meta-model  (CWM),  described  by 
the  Meta  Object  Facilities  (MOF).  This  possibility,  how¬ 
ever,  was  not  applied  in  the  underlying  project  so  far. 
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Figure  1:  Activity  Diagram 


In  the  second  phase,  the  actions  of  the  use  cases  are  com¬ 
bined  to  vignettes  and  scenarios.  This  allows  definition  of 
metrics  for  each  of  the  tasks  in  the  context  of  the  opera¬ 
tional  environment. 

This  phase  is  a  technical  refinement  of  the  CIM  into 
more  computer  or  simulation  oriented  views.  The  results 
are  object  and  type  hierarchies,  action  lists,  sequences  dia¬ 
grams,  and  other  artifacts  of  UML  that  describe  a  PIM. 
Figure  1  shows  an  example  of  an  activity  diagram  describ¬ 
ing  a  vignette  of  infantry  engagements  in  an  urban  envi¬ 
ronment. 

The  resulting  hierarchies  are  captured  in  UML,  but 
they  have  been  proven  to  support  the  communication  with 
military  experts  as  well.  All  tasks  can  now  be  described 
with  metrics.  Accuracy  and  resolution  are  decided  based 
on  military  expertise,  not  on  technical  constraints. 

The  result  of  this  phase  is  a  description  of  the  scenar¬ 
ios  or  vignettes  that  should  be  used  to  evaluate  the  aspects 
of  the  system  under  test.  An  easy  book-keeping  check  can 
make  sure  that  all  use-cases  of  phase  1  are  considered  in  at 
least  one  vignette  in  the  PIM.  Also,  each  role  must  be 
mapped  to  an  object.  If  a  complete  MDA  approach  is  used 
for  the  support,  all  objects  are  converted  into  elements  of 


3.3  Identifying  applicable  Simulation  Services 

As  implied  by  the  methods  and  processes  introduced  in 
section  2.2,  so  far  only  operational  requirements  were  used 
to  define  what  should  be  used  to  evaluate  a  system.  In  the 
third  phase,  the  simulated  systems  and  capabilities  are  used 
to  identify  applicable  simulation  systems. 

The  requirement  is  that  models  must  present  their  abil¬ 
ities  in  form  of  a  PIM.  The  PIM  defines  a  model’s  ability 
to  model  systems,  capabilities,  and  activities.  Concepts, 
properties,  and  processes  need  to  be  made  transparent.  The 
advantage  of  using  UML  artifacts  is  that  it  is  possible  to 
make  the  system  transparent  while  protecting  the  intellec¬ 
tual  property  of  technical  details  behind  the  implementa¬ 
tion. 

These  PIMs  look  very  similar  to  the  artifacts  produced 
in  the  last  phase.  Standardization  across  the  armed  forces 
will  support  alignment.  In  particular,  organizations  should 
name  the  same  objects  and  processes  identically  and  con¬ 
sistently,  using  these  definitions  to  tag  data  describing  the 
represented  concepts,  properties,  and  processes.  Standards 
like  the  Military  Scenario  Definition  Language  (MSDL) 
and  the  Coalition  Battle  Management  Language  (C-BML) 
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support  potential  solutions  to  this  challenge.  A  common 
data  administration  of  M&S  and  command  and  control 
would  be  helpful  as  well. 

In  any  case,  the  PIMs  of  the  simulation  systems  can  be 
used  to  find  out  which  elements  of  the  PIM  of  the  scenario 
and  the  vignettes  can  be  represented  or  simulated  and  for 
which  accuracy  and  resolution  support  the  metrics  identi¬ 
fied  by  the  military  expert.  The  systems  engineering  proc¬ 
ess  supports  several  objectives: 

•  Minimize  the  number  of  supporting  simulation 
systems  that  represent  the  scenario 

•  Minimize  the  costs  of  obtaining  the  simulation 
systems  and  supporting  data 

•  Maximize  the  use  of  simulation  system  under  go¬ 
vernance  of  the  project  manager 

•  Maximize  the  acceptance  of  systems,  etc. 

3.4  Preparing  the  Federation 


mulation  systems.  When  being  mapped  to  a  HLA  federa¬ 
tion,  the  attribute  used  in  the  Federation  Object  Model 
needs  to  be  identified  as  well. 
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The  result  of  the  last  phase  can  most  easily  be  visualized  as 
a  “colored”  PIM.  Each  object  and  each  activity  is  tagged 
with  the  simulation  system  information  that  can  be  used  to 
represent  it.  Some  objects  and  activities  represent  general 
concepts,  such  as  soldiers  and  tanks,  and  they  are  likely  to 
be  found  in  many  systems.  Other  features  are  very  special, 
such  as  waveforms  for  special  communications,  and  only  a 
few  simulation  systems  will  provide  them. 

The  colored  PIM  can  now  be  used  to  support  the  deci¬ 
sions  on  which  system  should  represent  which  objects  and 
activities.  This  decision  is  triggered  by  the  objectives  enu¬ 
merated  at  the  end  of  the  last  subsection.  The  optimum  for 
the  analysis  would  be  to  maximize  the  coverage  of  opera¬ 
tional  requirements,  but  other  constraints  -  such  as  time, 
funding  available,  or  security  concerns  of  model  providers 
-  can  limit  the  feasible  solution.  However,  no  matter  what 
motivates  the  ultimate  selection  of  models,  it  is  very  likely 
that  at  least  two  models  are  selected  that  need  to  be  feder¬ 
ated  to  provide  all  necessary  capability.  Only  in  rare  cases, 
everything  is  provided  by  one  model,  and  no  federation 
support  is  needed.  Whenever  and  activity  connects  two  ob¬ 
jects  hosted  in  different  systems,  or  whenever  properties 
needed  to  support  the  activities  or  the  identified  metrics  for 
one  object  are  provided  by  different  systems,  a  federation 
is  needed  to  handle  the  interactions  and  updates. 

The  patterns  supported  by  MDA  to  move  from  PIM  to 
PSM  support  integration  with  applicable  middleware. 
Without  limiting  the  general  applicability,  Figure  2  shows 
an  update  of  attributes  as  supported  by  IEEE1516.3  as  an 
example.  Whenever  an  attribute  needs  to  be  updated,  this 
sequence  of  calls  to  the  RTI  and  resulting  callbacks  need  to 
be  programmed. 

Whenever  an  attribute  of  an  object  is  owned  by  an¬ 
other  simulation  system  in  the  colored  PIM,  the  sequence 
shown  in  Figure  2  will  be  generated.  The  placeholders  in 
the  pattern  are  replaced  by  the  representing  objects  and  si- 


Figure  2:  Updating  Attributes 

In  the  same  way,  alternative  middleware  solutions  can  be 
supported,  such  as  mapping  to  PDU  of  the  IEEE  1278  DIS, 
or  objects  and  related  methods  within  the  object  model 
used  by  the  Test-  and  Training  Enabling  Architecture 
(TEN A).  The  use  of  web  services  is  another  option.  Fur¬ 
thermore,  mixed  strategies  can  be  supported,  such  as  using 
the  Extensive  Markup  Language  (XML)  file  based  MSDL 
for  initialization,  the  HLA  based  update  of  attributes  and 
sending  of  interactions  for  simulation  based  information 
exchange  during  runtime,  and  web  service  based  informa¬ 
tion  exchange  with  C2  systems  based  on  C-BML. 

Another  more  conservative  application  is  the  defini¬ 
tion  of  stubs  for  information  exchange  requirements  to  be 
enhanced  by  the  implementing  simulation  systems.  If  a  fu¬ 
ture  simulation  shall  replace  one  of  the  current  systems,  the 
interface  does  not  change.  In  fact,  the  initial  simulation  can 
test  the  federation  and  perform  preliminary  analysis.  When 
the  replacement  simulation  is  implemented,  it  federates  us¬ 
ing  the  same  interfaces.  The  MDA  pattern  identifies  ex¬ 
actly  what  elements  and  procedures,  methods,  and  call¬ 
backs  need  to  be  supported. 

4  AN  APPLICATION  EXAMPLE 

As  mentioned  in  the  introduction  to  this  paper,  the  systems 
engineering  process  was  designed  while  supporting  PEO 
Soldier.  PEO  Soldier  wishes  to  use  simulation  models  to 
support  the  procurement  process.  Because  no  one  model 
can  currently  support  PEO  Soldier’s  procurement  tasks, 
they  are  undertaking  an  integrated  approach,  extending  and 
federating  existing  models. 

In  the  example  task  supported  by  the  project  team,  the 
task  was  to  evaluate  different  infantry  soldier  architectures 
in  urban  environments. 


1301 


Tolk,  Litwin  and  Kewley 


1.  PEO  Soldier  decided  that  the  following  essential 
tasks  will  be  sufficient  for  a  first  evaluation: 
transport  in  a  HMMVV,  direct  fire  engagement 
with  insurgents  on  the  top  of  a  roof,  clearing  a 
house  in  search  of  at  least  one  enemy  inside,  indi¬ 
rect  and  direct  fire  from  hostile  forces  that  will  re¬ 
sult  in  a  call  for  fire  to  a  supporting  mortar  unit. 

2.  The  resulting  scenario  activities  are  captured  in 
Figure  1.  The  objects  were  the  roles  of  the  use 
cases.  The  resulting  PIM  could  be  used  to  identify 
two  models  that  if  used  in  conjunction  provide  the 
desired  capability  and  metrics  for  PEO  Soldier: 
One  Semi-Automated  Forces  (OneSAF)  and  In¬ 
fantry  Warrior  Simulation  (IWARS).  While  One¬ 
SAF  provides  the  frame  for  the  scenario,  IWARS 
provides  the  high-resolution  models  to  evaluate 
the  effects  of  soldier  equipment  such  as  body  ar¬ 
mor  and  night  vision  goggles. 

3.  The  scenario  activities  were  separated  into  One¬ 
SAF  activities  and  IWARS  activities.  Wherever  a 
crossover  shows  up,  information  needs  to  be  ex¬ 
changed.  Figure  3  shows  the  activities  side  by 
side.  Figure  4  shows  the  derived  view  for  indirect 
fire  between  OneSAF  and  IWARS  entities  minus 
RTI  interface. 


Figure  3:  Activities  in  OneSAF  and  IWARS 


Figure  4:  Indirect  Fire  Sequence 

4.  PEO  Soldier  decided  to  base  the  on-line  coupling 
of  OneSAF  and  IWARS  on  HFA  as  the  interop¬ 
erability  standard.  They  used  a  federation  object 
model  (FOM)  developed  as  part  of  the  Modeling 
Architecture  for  Research,  Experimentation,  and 
Technology  (MATREX)  program  as  the  informa¬ 
tion  exchange  model.  Therefore,  the  information 
exchange  requirements  resulting  from  the  PIM 
mapping  in  step  3  had  to  be  mapped  to  RTI  calls 
and  the  use  of  classes  and  interactions  with  attrib¬ 
utes  and  parameters  defined  in  the  FOM.  For  ex¬ 
ample,  the  “call-for-fire”  activity  had  to  be 
mapped  to  a  “call  for  fire”  interaction  as  defined 
in  the  MATREX  FOM. 

The  result  in  the  project  was  twofold: 

•  The  way  the  two  simulation  systems  work  to¬ 
gether  and  orchestrate  their  activities  is  now  cap¬ 
tured  in  standardized  form. 

•  The  artifacts  can  be  used  to  generate  software 
stubs  to  allow  the  easy  migration  to  alternative  so¬ 
lutions  (including  auto-generating  code  with 
tools). 

Even  if  only  used  for  documentation,  the  systems  en¬ 
gineering  process  supports  a  better  understanding.  Within 
the  project,  we  observed  many  discussions  regarding  which 
simulation  specific  attributes  to  map  to  and  from  the  FOM 
elements.  Just  having  the  common  representation  of  ob¬ 
jects  and  activities  and  sequences  facilitates  the  discussion. 
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5  A  COMMON  APPROACH  FOR  ACQUISITION, 
DEVELOPMENT,  TESTING,  TRAINING,  AND 
OPERATIONAL  SUPPORT 

Another  aspect  goes  beyond  the  project  that  sponsored  this 
work.  ODU  and  VMASC  are  supporting  a  variety  of  or¬ 
ganizations  working  in  the  domains  of  acquisition,  devel¬ 
opment,  testing,  training,  and  operational  support.  All  of 
these  organizations  use  M&S  in  one  way  or  the  other  to 
support  measuring  effects  and  capabilities.  However,  there 
is  no  sufficient  framework  established  to  ensure  the  align¬ 
ment  of  assumptions  and  constraints.  The  PIMs  as  used  in 
this  paper  can  be  a  significant  management  help.  The  ex¬ 
ample  of  the  metrics  shall  demonstrate  the  potential.  As 
described  in  sections  2.1  and  3.2,  metrics  are  defined  in  the 
context  of  four  elements  -  mission,  system,  evaluation,  and 
data  -  as  shown  in  Figure  5: 
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Figure  5:  Metrics 


procure.  Furthermore,  if  a  new  metric  is  successfully  used 
in  real  world  operations,  it  should  be  used  for  future  pro¬ 
curements  and  testing  as  well. 

The  same  is  true  for  the  PIMs  derived  from  the  METL, 
as  well  as  for  the  PIMs  describing  scenarios  and  vignettes. 
It  makes  no  sense  to  have  different  “business  views”  in  dif¬ 
ferent  domains  with  respect  to  the  same  mission  essential 
tasks. 

6  SUMMARY 

Using  a  systems  engineering  process,  operational  require¬ 
ments  and  technical  constraints  can  both  be  integrated  into 
an  MDA  based  framework  supporting  project  managers, 
model  developers,  and  evaluators  in  a  consistent  way.  No 
information  is  lost  in  the  translation  process,  so  that  man¬ 
agers  and  engineers  can  use  the  same  framework  to  com¬ 
municate  their  challenges  and  solutions  without  violating 
constraints  and  areas  of  responsibility  of  other  team  mem¬ 
bers. 

The  BOM  standard  (SISO  2006)  is  not  fully  suppor¬ 
tive  in  all  aspects  of  the  MDA,  but  its  application  enables  a 
significant  portion  of  documenting  M&S  components  and 
services  as  required  for  selection  and  orchestration. 

The  recommended  solution  enables  project  manage¬ 
ment  of  simulation  based  acquisition  and  supports  the 
alignment  of  procurement,  development,  test,  and  training 
by  introducing  a  common  view  derived  from  operational 
needs,  including  a  set  of  consistent  metrics. 
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•  The  mission  and  means  framework  sets  the  opera¬ 
tional  context  for  the  mission  essential  task  that  is 
measured  by  a  metrics.  This  defines  what  and  why 
something  has  to  be  accomplished. 

•  The  system  to  be  evaluated  (or  the  system  under 
test)  is  the  system  and  its  capability  currently  de¬ 
livering  the  functionality  needed  to  conduct  the 
mission  essential  task.  This  defines  who  is  doing 
the  task,  and  also  how. 

•  The  formula  used  to  compute  a  value  for  the  met¬ 
rics  is  an  element  on  its  own. 

•  Finally,  the  collected  data  belong  to  the  metrics 
context  as  well. 

This  form  of  metrics  was  first  recommended  by  Jack 
Sheehan  and  Dr  Paul  Dietz  for  the  US  Army  Test  and 
Evaluation  activities.  The  idea  is  not  limited  to  testing  but 
applicable  in  all  domains.  For  example,  it  would  make  no 
sense  if  metrics  used  in  the  operational  testing  of  systems 
are  different  from  those  used  to  decide  which  system  to 
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